Two Perspectives on Learning Rich Representations from Robot Experience

This position paper describes two approaches towards the representations that a robot can learn from its experience. In the first approach, the robot learns models for reasoning about human-interpretable aspects of the environment, for example models of space and objects. In the second approach, the robot incrementally learns predictions for the consequences of performing policies, where a policy is any experimental procedure that the robot can perform. These two approaches correspond closely to the ideas of a scientific model and an experimental prediction, and ideally the benefits of both can be accessible to a robot. The different prerequisites needed to support reasoning and learning have led to different forms for representations learned from robot experience. The desire to enable a robot to reason about its environment in terms familiar to people has led to procedures for learning representations from a robot’s low-level experience that support inference for models of space (Pierce and Kuipers 1997), objects (Modayil and Kuipers 2007), and even communication (Steels 1998). The desire to accurately predict the temporally extended consequences of a robot behaviour has led to statisticallysound fully-incremental learning algorithms (Sutton 1988; Maei and Sutton 2010). These two perspectives correspond to the complementary capabilities of scientific models and scientific experiments. A scientific model describes some aspect of a system and ignores other aspects as being outside the scope of the model. Scientific models can be descriptive (e.g. a classification of living organisms into genus and species), or they can simulate system dynamics based on a particular description (e.g. celestial mechanics). For many computational purposes, the most useful capability of a model is to simulate some aspect of the dynamics. As an example of such models, the SLAM approach to robot mapping relies on observation models (assigning likelihoods to observations given the robot’s pose and the map) and motion models (generating samples for the robot’s next pose based on the motor commands). Any particular model of the physical world will be imperfect, this is true for scientific models and for the computational models available to a robot. Having access to multiple models can mitigate the limitations of any particular model, for example a robot with causal, topological, and metrical maps can use any of these models to support navigation (Kuipers 2000). However, building large systems with interacting models is challenging because of differing semantics at the interfaces between models. Human intervention is often required to orchestrate and debug the interactions between models that lack a robot-interpretable semantics. The scientific experiment complements the scientific model, by measuring a single aspect of the world through a carefully specified experimental procedure. A robot with the ability to predict the result of performing a procedure has a limited but concrete piece of knowledge. Moreover, because the result can be observed, this knowledge can be verified by a robot, and it can be the target of a learning algorithm. Even when a model is used to provide predictions about an experiment, measurements of a precise scientific procedure can exhibit variations that the model fails to predict. For example, a robot might move at different speeds on asphalt or on ice, but a standard motion model would not predict this variation. A disagreement between two models can be used to suggest a new experiment. Conversely, generalizing from particular experiments might suggest a useful form for a model. The interaction between models and experiments provides the foundation of scientific knowledge. This interaction has been explored as a foundation for learning in children (Gopnik, Meltzoff, and Kuhl 1999), and it could provide similar insights for structuring a robot’s knowledge. An initial concern is whether such an approach is tractable on a robot, namely whether models or predictions might be learned by a robot. The remainder of this paper surveys some results that suggest that both models and predictions can be learned by a robot from its sensory-motor experience.